The problem of structural optimization is to find improved or optimal structural designs from a space of possible structural designs. There is a means to evaluate the performances of each candidate design. A search algorithm traverses the design space getting feedback from the performance measures in order to propose new designs. To traverse the design space, the algorithm applies “operators” that take one or many designs and produce another design from those designs. Other problem domains outside of circuit design include design of neural networks, Bayes nets, industrial designs (e.g. chairs), and schedules.
One way to define the space of possible designs is to begin with initial designs and include all the designs reachable through the application of a finite number of operators. In traditional optimization problems, the design space typically only contains parameters, not structures. This can be called the problem of “parameter optimization.” Operators in parameter optimization only vary the parameters of the design, not the structure. An example parameter optimization problem in neural network design is to find the values for all the weights of the neural network that give the lowest mean-squared error in approximating the neural network to fit target data. In structural optimization, there are also one or more structural operators. These operators actually alter the structure of a candidate design. A simple example of a structural operator from the domain of neural network design is to merely remove a randomly chosen hidden node.
Structural optimization has benefits over parameter optimization for the reason that the design space is broadened to include different possible structures. With a broader design space than parameter optimization, there is the potential for even better designs to be achieved.
Unfortunately, structural optimization is typically very inefficient because it takes large amounts of computational effort to optimize structures. A major reason for the inefficiency is that when structural changes are made (i.e. structural search operators are applied), there is typically a low probability of improving the performance measure of the design.
Generally, structural operators are “non-smooth” operators because they typically result in large changes in design behaviour. A non-smooth operator has typically been designed without consideration of the effects that parameters have on changes in behaviour in the specific domain of application. Applying a non-smooth operator typically leads to non-negligible changes in behaviour. So, on average, expected changes in behaviour are high. Large changes in behaviour usually lead to large changes in performance measures. The larger an expected change in a performance measure, the lower the expected probability of successfully improving the measure.
In the neural network design domain for example, consider the application of the non-smooth operator in which a hidden node is randomly removed. In all likelihood, the network's behaviour in terms of flow of signal strengths will be radically changed. A large change in the signal behaviour likely means a large change in the subsequent performance measure, e.g. a large change in mean-squared error. With such a large change in mean-squared error, there is a small likelihood that the change has been for the better. Successful changes to the network's structure are thus rare.
The main benefit of structural optimization over parameter optimization is the ability to create better designs, given enough time, because the scope of possible designs is larger. Unfortunately, structural optimization is too computationally expensive for many applications.